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ABSTRACT 



A system and method are provided for supporting video 
browsing over a communication network such as the 
Internet/World Wide Web. A graphical user interface is 
provided through a client software tool such as a Web 
browser. A client/user selects a video data object stored at a 
remote server. A set of points within the object are displayed 
at the client's graphical user interfa ce dis play , as 
representations, preferably thujnb.naiLimages, of the -points 
within the object. The user selects an interval defined by the 
representations, preferably by using the graphical user inter- 
face to select two of the representations. The two selected 
representations delimit the beginning and end of a portion of 
the video object. Responsive to this selection, that portion of 
the video object is downloaded and displayed. 

48 Claims, 4 Drawing Sheets 
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VIDEO STORY BOARD USER INTERFACE Bernstein et al., U.S. Pat. No. 5,297,249, issued Mar. 22, 

FOR SELECTIVE DOWNLOADING AND 1994; and 

DISPLAYING OF DESIRED PORTIONS OF Lewis, U.S. Pat. No. 5,355,472, issued Oct. 11, 1994; 

REMOTE-STORED VIDEO DATA OBJECTS all of which are assigned to International Business Machines 

FIELD OF THE INVENTION 5 Co[ P oratioD ' 



The invention generally relates to the field of the storage, 
viewing, and handling of video data objects. More 
specifically, the invention relates to computer systems for 
browsing such video data objects. The invention has par- 
ticular applicability to viewing and browsing video data 
objects provided from a remote repository over a network 
such as the Internet. 

BACKGROUND OF THE INVENTION 
GLOSSARY OF TERMS USED 

While dictionary meanings are also implied by certain 
terms used here, the following glossary of some terms may 
be useful. 

Internet ("the Net"): The connection system that links 
computers worldwide in a network. 

TCP/IP: Transmission Control Protocol/Internet Protocol. 
A packet switching scheme the Internet uses to chop, route, 
and reconstruct the data it handles, from e-mail to video. 

World Wide Web (WWW. "the Web"): The Internet's 
application that lets people seeking information on the 
Internet switch from server to server and database to data- 
base by clicking on highlighted words or phrases of interest. 
An Internet Web server supports clients and provides infor- 
mation. 

Home page: A multi-media table of contents that guides a 
Web user to stored information on the Internet. 

Servers A machine (computer) which performs a task at 
the command of another machine ("client"). In the context 
of the present invention, a server's primary function is to 
facilitate distribution of stored information over the Web. 

Client: A machine which provides commands to a server, 
ana is serviced by the server. Typically, a client machine is 
operated by an end user, and functions responsive to user 
commands. 

Web Browser: A program running on a user-operated 
client computer. When a user "surfs" the Web using a 
browser, the browser acts as an Internet tour guide, allowing 
the client machine to display pictorial desktops, directories 
and search tools supported by the server. 

URL: Universal Resource Locator, a Web document ver- 
sion of an e-mail address, in character string form, which 
uniquely identifies a document, application, or tool available 
over the Web. 

Hyperlink: A network addressing tool embedded in a 
user-understandable displayed and/or highlighted item, such 
as a word, phrase, icon or picture. A URL can be accessed 
by means of its corresponding Hyperlink. When a user on a 
client machine selects the highlighted hyperlink through the 
user interface, the underlying item is then retrieved to the 
client supporting a Web browser. 

HTTP Hypertext transfer protocol: Hypertext transfer 
protocol. The character string "http:" at the beginning of a 
URL indicates that the document or file designated by the 
URL contains hyperlinks defined according to the HTTP 

Hypertext Markup Language (HTML): HTML is the 
language used by Web servers to create and connect docu- 
ments that are viewed by Web clients. HTML uses Hypertext 
documents. Other uses of Hypertext documents are 
described in the following U.S. Patents: 

Bernstein et al., U.S. Pat. No. 5,204,947, issued Apr. 20, 
1993; 



BACKGROUND OF THE INVENTION 

In recent years, the technologies of video data 

10 compression, storage, and interactive accessing have con- 
verged with network communications technologies, to 
present exciting prospects for users who seek access to 
remotely stored multimedia information. 
In the area of network communications technologies, 

15 particularly exciting has been the recent prominence of the 
Internet and its progeny, the World Wide Web, The Internet 
and the Web have captured the public imagination as the 
so-called "information superhighway/' Accessing informa- 
tion through the Web has become known by the metaphori- 

20 cal term "surfing the Web." 

The Internet is not a single network, nor does it have any 
single owner or controller Rather, the Internet is an unruly 
network of networks, a confederation of many different 
networks, public and private, big and small, whose human 

25 operators have agreed to connect to one another. The com- 
posite network represented by these networks relies on no 
single transmission medium. Bi-directional communication 
can occur via satellite links, fiber-optic trunk lines, phone 
lines, cable TV wires and local radio links. 

30 * To this point the World Wide Web (Web) provided by the 
Internet has been used in industry predominately as a means 
of communication, advertisement, and placement of orders. 
The World Wide Web facilitates user access to information 
resources by letting people jump from one server to another 

35 simply by selecting a highlighted word, picture or icon (a 
program object representation) about which they want more 
information — a maneuver known as a "hyperlink". In order 
to explore the WWW today, the user loads a special navi- 
gation program, called a "Web browser" onto his computer. 

There are a number of browsers presently in existence and 
in use. Common examples are Netscape's "Navigator," 
Microsoft's "Explorer," Mosaic's browser, and IBM's "Web 
Explorer." Browsers allow a user of a client to access servers 

45 located throughout the world for information which is stored 
therein. The information is then provided to the client by the 
server by sending files or data packets to the requesting 
client from the server's storage resources. 
Part of the functionality of a browser is to provide image 

50 or video data. Web still image or video information can be 
provided, through a suitably designed Web page or interface, 
to a user on a client machine. Still images can also be used 
as Hypertext-type links, selectable by the user, for invoking 
other functions. For instance, a user may run a video clip by 

ss selecting a still image. 

However, video data objects are very large, or, to put it 
more precisely, the quantity of data per unit time in a 
real-time viewing of a video data object is large. As a 
consequence, access by a user to a desired video data object 

60 is subject to data throughput constraints. The present state of 
the art makes it impracticable to provide more than a few 
tens of seconds of real-time video over the Internet with a 
response time that will be satisfactory to a user. 
Also, many video objects are stored in compressed 

65 format, and must be decompressed for playback or brows- 
ing. Formats such as Motion JPEG and QuickTime com- 
press video on a frame -by- frame basis. Accordingly, access 
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to a desired part of the video is relatively easy: a time stamp In a typical environment, in which the user is an Internet/ 

or token may be used for direct access to the desired point World Wide Web user, and the video data object is stored in 

in the video. a remote repository accessible through a server, the step of 

In the MPEG compression format, however, compression providing the representations includes performing suitable 

is done over the time domain as well, so decompression of 5 operations to identify points within the video data object, for 

a significant amount of real video time may be required for which representations are to be provided. Generally, an 

access to a given point. automatically operating tool is used to identify the points, 

Therefore, multimedia and communication systems for and the points are chosen based on a predetermined crite- 

providing users with access to video data objects, for rion. A preferred criterion is the presence of scene cuts, in 

browsing, searching, etc., must grapple with the problem of 10 WQ i cn fa c con tent of adjacent video images changes rela- 

providing video data in a manner which best utilizes the ^ yc]y drastically. However, where scenes are relatively 

available throughput to provide video data in a form which leagthv> frames mc sccnc> { e ? bctwccn me cuts at thc 

is most useful to the user. beginning and end of the scene, are also provided. 

With this design obi ective m mind, let us now consider the « • . r • -jj-ij- * 

imuii* B j , Then, a user interface is provided, including means for 

state of the art in the technologies of video data - c „ . t Tt .\ , . ' « f j j 

compression, storage, and interactive accessing. Recent 15 allowin S * user to s f ci an m *™ X betwee ? ^ and f c< ? nd 

work has been done to make video material more available ones °f the temporal points. The user interface is preferably 

and usable over the Web. For instance, an article in the a graphical user interface (GUI), as is commonly made 

August 1995 issue of ADVANCED IMAGING, by Amy T. available through computer operating systems such as 

Incremona, titled "Automatically Transcribing and Con- IBM ' S OS/2 and Microsoft's Windows operating systems, 

densing Video: New Technology is Born", describes a 20 ^ smtable user mtelfiK * equipment, such as a video 

method for providing video having an accompanying textual screen and a mouse > are P referabl V used - 

index, such as audio narration or closed caption text. Still Finally, responsive to user selection of such an interval, a 

images are presented, along with a transcription of audio text subsequence of representations is provided, each represen- 

that accompanies the images (illustration on page 60). This tation of the subsequence corresponding with a respective 

information is provided in HTML format. Thus, a user can 25 temporal point in the video data object, each of the respec- 

take advantage of the temporal correspondence between tive temporal points falling between the first and second 

video shots and narration or closed caption text. To find a temporal points selected as discussed above, 

desired point in the video corresponding with a known point Therefore, by providing this capability, the Steele inven- 

in the text, the user performs a key word search for the tion allows a user to iteratively home in on the portion of a 

known point in the text. The result of this key word search lengthy, large video object that he/she is interested in. The 

is that the desired point in the video is reached. Steele invention advantageously eliminates the need to use 

Additionally in Shahraray et aL, "Automatic Generation text accompanying the video object as a crutch, as was done 

of Pictorial Transcripts of Video Programs", SPIE Vol. 2417, in the prior art discussed above. 

pp. 512-518, there is described an automatic authoring 35 However, once a user identifies a desired segment within 

system for the generation of pictorial transcripts of video a video, he/she will likely want to view the segment, 

programs which are accompanied by closed caption infor- Accordingly, the segment must be downloaded, 

ma tion. The system employs a table having a series of rows, Conventionally, users have had to download the entire 

each row containing a pointer to a location of an image, and video, and run it from the beginning until the desired 

another pointer to the beginning of a text segment related to ^ segment is reached. This conventional operation, as taught 

the image. A viewing window for a GUI display in FIG. 4 in the Steele patent, has been inconvenient for the users, both 

of Shahraray et aL, shows a video image 2, a closed caption in terms of download time and in terms of browsing of 

text subtitle area 4, and a basic user control area 6. The basic desired segments within a larger video, 

user controls include a "Seek" slider 8. More recently, streaming technology has become 

Accordingly, the state of the art allows for user access to 45 available, in which a full download followed by playback is 
video information based on associated text. However, a replaced by download concurrent with playback, or down- 
more general method for accessing video, uot provided by load in shorter segments, and playback concurrent with 
the prior art, would sever the tie between video images and download, perhaps with pauses if the segments do not keep 
accompanying audio narration or closed caption text. up with real time playback. However, such streaming still 

Co-pending, co-assigned U.S. patent application Ser. No. 50 provides the drawback that a user does not have random 
08/581300, filed Dec. 28, 1995, Steele, "Method and Sys- access to the content of a video, but must view the video 
tem for Video Browsing on the World Wide Web" from the beginning until a section of interest is reached, 
(incorporated herein by reference) describes a system and Therefore, there remains a need for further techniques for 
method for providing convenient user access to a stored improving the user's ease of access to desired video mate- 
video object without requiring that access to the video object 5S rial, 
be keyed with any text or other accompanying indicia, not 
actually part of the video itself. 



SUMMARY OF THE INVENTION 



The Steele method for displaying, on a user terminal, It is therefore an object of the present invention to 

video data object information pertaining to a stored video facilitate convenient user browsing of video objects stored at 

data object, the stored video object including a temporal eo a remote reposit ory, such as a remote ne twork server, 

beginning point, a temporal end point, and temporal inter- It isTTurther object of the invention to reduce the 

mediate points therebetween, comprises the following steps: download time through which a user must wait before he'/She 

First, a sequence of representations of points within the can view a desired portion of a video data object, 

video data object, preferably still images, is provided to the It is a further object of the invention to allow a user to 

user through a user interface. Each representation or still 65 conveniently view only a dgsircd portion of a larger video 

image that is provided corresponds with one of the temporal o bject, rather than requiring that the user watch the video 

points in the video data object. object from the beginning up to the desired segment. 
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To achieve these and other objects, there is provided, in by the displayed representations. Numerous ways of doing 
accordance with the invention, a system and method for this are given in the detailed description. One example, 
facilitating user examination and browsing of video data given here for summary purposes, is to select two frames, 
object information pertaining to a stored video data object not necessarily adjacent to each other in time. The two 
(hereinafter referred to as an "Object"). For convenience if 5 selected frames delimit an interval within the video repre- 
discussion, an Object will be characterized as including a sen ted by the story board. For many GUI-type user inter- 
beginning point, an end point, and intermediate points faces as discussed above, the user selection my be made, for 
therebetween. instance, by clickin g on two o f the thumbnail images ofj he 

The invention, in its incarnation as a method, comprises stor y board. 

the steps now to be listed. (The invention may also be 10 "Responsive to the se lectio n of the4wo frames, the inte rval 

embodied in a system comprising means for performing the of the video, delimited by those frames , is downloa ded and 

steps listed.) providedr~&ince only that interval, rather than the whole 

First, a sequence of representations of points within the 06]ert; is downloaded, the download time is advantageously 

video data object, preferably still images, is provided to the reduced. 

user through a user interface. Each representation or still 15 Therefore, by providing the capability of selecting and 

image that is provided corresponds with one ofjhe temporal downloading only a desired portion of a larger Object, the 

points in the video data object. Preferably, the representa- invention allows a user to efficiently obtain the portion of a 

tions are thumbnail images of stills from the video Object. lengthy, large video object that he/she is interested in. The 

The stills are obtained through a selection technique not invention advantageously eliminates the lengthy download 

part of the present invention. Stills may be selected through 20 time which would be required if the entire video were 

known techniques. Preferably, the sH]]s r when scanned in downloaded. 

sequence hv ifm lifte r's, eyes, fihould piye the user a sense of ^ ^er will find the invention useful when he/she wants 

megoatgnXa t the vfiteo . T his may be, but is not necessarily, t0 qu i c kly scan, or preview, the gist of the video without 

achieved by spacing the stills uniformly in time, such as at actually watching the full motion data from beginning to 

regular time intervals. 25 end 

The stills are arranged chronologically, preferably in As a further time saving, which is advantageous even over 

rows. Accordingly, by scanning the thumbnail images m conventional streaming playback, the user need not watch a 

order, the user can get a quack sense of the progression of the lengthy video 0bject from ^ begirmiDg) if ae/she ^ only 

content of the Object. It is therefore deemed appropriate to 3o interested ip a poition 0 f me video. Rather, by judiciously 

refer to the set of displayed thumbnail images as a "story choosing me desired interval, the user can begin viewing at 

board." tne desired spot, within the Object. Thus, the invention 

The invention has applicability to any system or archi- advantageously facilitates video browsing, 

lecture in which a user, at an interactive terminal, clknt Whfle ^ inventkm i& primarily disclosed as a me thod, it 

node, etc., communicates with a video object repository. The 35 ^ be understood b a D of ordinary ^ m the ^ that 

invention is conceived, however, as having particular apph- ^ m such as a conventional data processor, includ- 

cabihty to the Internet. Ina typical environment, then, the & me gt ft connectin bus> 

user is an Internet/World Wide Web user, and the video data ^ riate componentS) could be programmed or 

object is stored m a remote repository accessible through a otherwise designed to faciljtate ^ practice of the method of 

server - 40 the invention. Such a processor would include appropriate 

Tne step of providing the representations includes per- program means for executing the method of the invention, 

forming suitable operations ^'^^^^^ ^ „ article of manu f a cture, such as a pre-recorded 

operations, to identify points withm the video data object, ' . ' * „„•*!, 

* . t J ^ u -j j a p ui^ disk or other similar computer program product, tor use with 

for which representations are to be provided. As a preferable . * , : r. / ' 

t . K . * j- *u ; iu A a data processing system, could include a storage medium 

alternative, the step of providing presupposes that the iden- - * & 7 > P tU A 

t .„ , . ' . r r . , " Jf. . « rt * t „ 45 and program means recorded thereon for directing the data 

tilled points have previously been determined, prior to * r t . c .... . ,. & tU , f 

*• e • *■ j * a i , # l processing system to facilitate the practice of the method of 

operation of the mvention, and are stored along with the f f, J j, , 

Ob'ect itself invention. It will be understood that such apparatus and 

I , * . , r, • . ..i. i articles of manufacture also fall within the spirit and scope 

Several additional aspects of the mvention, which are also - . . . 

„ „ r „ , . t . ' . . . oi trie invention, 

preferred, are as follows: Exactly where the identified points 50 

are located within the Object is not essential to the mvention. BRIEF DESCRIPTION OF THE DRAWINGS 

The next step of the method of the invention is that a uger i . , , «. , t . , . „ . , 

. 4 c ■ i _i • i j ■ r 11 „<TT7T FIG. 1 is a system block diagram of a typical distributed 

lnterfacejs provided, including means for allowing a user to 3 .... • , , . . i .. , 

~ imi r y -~ — Z — jr~ computer system which has internal and external networks, 

select an interval between first and second ones or th e . , / , . . i- . * . 

' 1 Q ts 'j'^g user int erf ce is referabl a raphica l including the Internet, to connect clients to World Wide Web 

empora poin i a p y g p 55 servers and otber server c the client systems being capable of 

user interface (GUI), as is commoQly made available . • . • • 

4 , , . ' • x , Ir ,w, ncn mcorporating the mvention. 

through computer operating systems such as IBM s OS/2 r b 

and Microsoft^ Windows operating systems. Also, suitable FIG - 2 is a flowchart showing the method of the invention, 

user interface equipment, such as a video screen and a FIG. 3 is an illustration of a user interface in accordance 

mouse, are preferably used. go with me invention. 

Where the invention is to be applied in an Internet/World FIG. 4 is an illustration of a user interface in accordance 

Wide Web environment, the GUI may be provided in the with the invention. 

form of a Web page, having suitable features for providing nPSPRTPTrnN hp thf PRFFFRRFH 

user information, including the story board, and for receiv- DESCRIPTOR N OF THE PREFERRED 

ing user input commands 65 EMBODIMENT 

Employing the user interface, the user selects an interval The invention may be practiced in connection with any 

in terms of the points in time, within the video, represented system in which a user operates a local computer system, 
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having a suitable user interface such as a GUI, for browsing The program run by the server 14, which incorporates the 
a video Object stored in a separate or remote repository. invention, may be provided to a general purpose computer 
It will be understood that all such systems, architectures, on a pre-recorded medium, as discussed above, such as a 
methods, etc., which would be deemed suitable for use by a floppy disk 15, in which case the novel and non-obvious 
person skilled in the art, fall within the spirit and scope of 5 features of the program code are pre-programmed onto the 
the invention. The scope of the invention, thus, also covers disk 15 for convenient sales and marketing, and for directing 
systems residing entirely a single, local site. Communica- operation of the server machine 14 in accordance with the 
lions would then take place over direct local connections, invention, 
and remote point-to-point network communication is omit- 
ted. For instance, a local file system at a single location may 1Q STORAGE OF VIDEO OBJECTS ACCORDING 
serve as a repository for video objects which a user might TO THE INVENTION 
want to access. Such a system would, by itself, contain all _ . . . . . . 4 

elements of the invention Id *«f <*ance ™* invention video objects are rep- 

However, the discussion which follows will focus on an t ^^^°^^^ h ^ 

exemplary application of the invention to the Internet/World c ^ °^ c f: ^ * P re u ferred embodiment employing the Web 

Wide Web environment. It will be understood that this 15 m HTML links, the representations of the temporal points 

exemplary discussion in no way limits the wider applica- are thumbnail images corresponding with the temporal 

bility of the invention. It will be understood, for instance, P° mts m me vldeo ° b J cct - ^ «nag« may be generated at 

that any "intranets" for distinct populations of users, which the r tim l e 1 a user *k<** A Vld u e0 ? b J c * for ^owsing, or 

may be separated from the overall Internet by means of P ref « abl y are generated beforehand, and stored along with 

"firewalls" of the like, fall within the spirit and scope of the 20 the ob J ect ltself • 

invention. The temporal points at which the representations are to be 
FIG. 1 illustrates a typical communication and processing taken ma V be in ^ suitable fashioQ * 11 fe Purred 
system, including nodes (computers, processors, etc.) and that the temporal points be selected at scene cut points in the 
communication media-therebetween, making up an infor- „ video object, ^own techniques for scene cut selection may 
mation delivery solution of a typical combination of 25 ° e used. See, for instance, a description given in U.S. patent 
resources including clients, se rvers, i nformation application Ser. No. 08/581,300 (referred to in the Back- 
repositories, and commu nicatio^nWnr ™iwml« Various g round ^ chon of * e P r u esent P atenl application). The tern- 
nodes, shown typically asXare also coupled to one another P° ral P oints mav be chosen at random Wlthin the Vlde0 
by various networks, shown generally as 7, which may object. 

include local area networks (LANs), wide area networks However, in another possible embodiment of the 

(WANs), etc. invention, the temporal points are evenly spaced throughout 

The discussion of FIG. 1 which follows will focus on the video ob J ccL 11 should be notcd here ^at "evenly 

certain individual components for clearer illustration and s P aced " 15 intended to have a subjective, rather than literal, 

description of the novel and non-obvious features of the 35 meaning. As will be described in detail below, the user 

present invention. In the system of FIG. 1, a client machine selects two of the temporal points, with the objective of 

10 includes a computer or other device (as discussed above), downloading and viewing the portion of the video object 

running a Web browser program. The program run by the ^ falk between the two temporal points. It is preferable, 

client 10, which incorporates the invention, may be provided from the standpoint of the system's look and feel to the user, 

to a general purpose computer by means of a commercial „ that the time duration of the selected video portion ("clip" or 

computer program product, such as a pre-recorded floppy "segment") be perceived as proportional to the number of 

disk 11 or other suitable computer-readable recording representations captured within the interval selected by the 

medium. In such a computer program product, the novel and user - Where ^ ^ Wl11 feel such a proportionality, the 

non-obvious features of the invention, implemented as pro- subjective criterion for "even spacing" is deemed to have 

gram code, are pre-programmed onto the disk 11 for con- 45 been ac hi evea1 . 

venient sales and marketing, and for directing operation of OPERATION OF THE INVENTION 
the client machine 10 in accordance with the invention. 

The client 10 is coupled, through a communication net- Giv ? n tbe foregoing comments, a description of the 

work 12 such as the Internet, to a Web server 14. Physical operation of a user interface according to the invention will 

access to the Internet is provided in conventional fashion. 50 now bc dcscrib cd. 

The general command protocols, etc., for exchanging mes- A preferred embodiment of the invention employs a client 

sages between the client 10 and the server 12, are also machine supporting a user interface utilizing HTML format, 

conventional. and further employs the World Wide Web. Some of the 

The client machine 10 may preferably be an intelligent functionality of the invention may be carried out at the 

computer system, such as an IBM PS/2 computer, an IBM 55 server However > the description which follows will be 

ThinkPad laptop computer, or an IBM RISC System 6000 readil y understandable from the vantage point of the user at 

workstation. Communication coupling made to the network ^ client machine. 

via a suitable communication interface software tool, such This description will take the operation of the invention in 

as IBM's OS/2 WARP Connect software product. chronological sequence. The description will be given with 

The Web browser in the client machine 10 may preferably 60 reference to a high level flowchart shown in FIG. 2. 

be the IBM Web Explorer software product, or equivalent A user begins by select ing a video o bject from a directory 

software tools such as the Netscape or Mosaic tools. This of stored objectsistepiO^ITl^ suitable 

computer system 10 is bi-directionally coupled with the fashion, based »n4mo^o graphical interface techniq ue s. The 

OS/2 WARP Connect facility over a line or via a wireless Web provides •RumerousTOiown formats, techni ques, etc!7Ior 

system to the server machine 14. The server machine 14 may 65 selecting objects: 

preferably be another IBM PS/2 computer, an IBM RISC For instance, responsive to an initial user command 

System 6000 workstation, or other similar system. . establishing contact with the remote repository, a menu of 
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video titles could be displayed. The user ma y then select a the spirit and scope of the invention, as given broadly in 

HftsirftH titl ft hy cjirkin ff nn it ( t yping its identifier into a field, terms of step 30. 

efc*T~_ Once the user has specified a desired segment of video, in 

Responsive to the user selection of a desired video object, accordance with the invention that video segment is down- 

the remote system obtains a set of representations of points * loaded (step 32). Since only the desired portion of the video 

within the video object (step 22). Preferably, the represen- f downloaded, the invention provides an advantageously 

tations are thumbnail images of stills from the video object, fas ' res P onse 10 { ^ rs selecUon ' bj . 

so that the user can scan hi^r eyes a 1on P the thumbnails lx ls ^cognized, however, that, where the video object is 

* t c t c • ' rtf iUa compressed in a time-domain format, such as MPEG, some 

and get a sense of the progressionojJne^onientJaction, . . ... .,, u 

— \ ciC — rr — u- — * tu.v u^i«„7u ™ additional processing time, at the repository, will be 

narrative, etc.) of the video object. This sense helps the user 10 , , , r . , 6 * a a 

\ ... - «... . j . . r . . . expended doing the decompression that is necessary to rind 

to determine which portion of the video object is of interest. ^ dcskcd of ^ ft may be ^ c ^ ^ ^ 

Note that the terms "representations," "thumbnails," finding is expedited by starting at a point in the MPEG- 

"stills," etc., will be used interchangeably. It will be compressed video prior to the beginning of the requested 

understood, however, that the spirit and scope of the inven- section. It may be expedient, in some applications, to 

tion extends broadly over all forms of representation that a 15 download a segment containing more than the video actually 

person skilled in the art would understand to be suitable for requested, but beginning at an expedient MPEG decompres- 

use. The various other, more specific, terms will be used for sion point. Then, the local system decompresses starting at 

illustrative purposes. that point, and discards whatever initial portion of the 

In accordance with the invention, it is these representa- 2Q decompressed video was not requested, 

tions which are downloaded in response to the user's selec- The particular technique of downloading the video is not 

tion of a desired video object. Advantageously, the set of essential to the invention. It is contemplated, however, that 

representations may be downloaded much more quickly than either a full download followed by playback, or streaming, 

the entire video object could be downloaded. are techniques that will be useful in implementing systems 

FIG. 3 is an illustration of a user interface according to the ^ incorporating the user interface of the invention, 

invention. On a graphical user interface, such as a Web page Finally, the segment is played back, at real time speed or 

24, a group of thumbnail images, generally shown as 26, are as otherwise desired, responsive to the user's command, 

displayed. The number of thumbnails will likely be enough FIG. 4 shows a playback interface, which is displayed after 

to fill a GUI screen or even greater than that. toe user makes the selection from the user interface of FIG. 

Where the number of thumbnails exceed the space pro- 30 l yP* ca * playback control interface, including controls to 

vided by the screen, a paging or scrolling feature may be allow * **** nonnal P la 5? ack 36 ' f f 1 fo ™ rd 38 ' 

provided. In FIG. 3., a scroller arrangement 28, including reverse playback 40 fast reverse 42, pause 44, stop 46 etc., 

scroll-up and scroll-down buttons, is provided. Responsive 15 provided. The video is shown in a display window 48. 

to user mouse clicks or the like, the displayed thumbnails Mso ^ a contro1 ma y be supplied to return the user to the 

may be scrolled by row, by page, etc. 35 selection interface of FIG. 3. Where a World Wide Web 

Returning to FIG. 2, the user, viewing a display such as us f*> * e ™* bxo ™" wiU ^ly supply forward 

that of Fia 3, selects an interval within the video object and backward buttons - In such a cas *> these ma ? be used * 

(step 30). This may be done simply by clicking on two of the Optionally, a video index timer 50 may be provided to 

thumbnails. Other techniques may also be used. For give the user a measurement of the realtime either withm the 

instance, the user may depress and hold a mouse button 40 selected segment or within the entire video Index values 

(preferably, the left button) while sliding the mouse from a ma y also be ^PP 1 ^ Wlth ^ thumbnails of FIG. 3. 

first thumbnail to a second one. Note ^at dis P la y systems employing the invention need 

Also, the user may have the option of selecting a single ™\ P 1 ^ audio. Audio is conveniently omitted. However, 

thumbnail, and entering a suitable commend to indicate that audio can be added separately, or the sound track can be 

the portion from the beginning to that thumbnail, or from 45 treated similarly to the way the video is treated That iSj dips 

that thumbnail to the end, is desired. of *°™ d can bc downloaded and played, in the 

. A , , . A . . same manner that some compact disk digital audio players 

Yet another option would be to select a single humbnail aUow ^ {q ^ h an ^ ^ fa ' 

and request aj gecified quannty of real time fr om that point short { n{s of ^ > a fast . forward B * 

For instance, the user might select a thumbnail and request perceivable aspects such as pitch, 

ten seconds of video^starting.at that point. ^ £ Me {Q home ^ Qn & 

Also, where a single representation is selected, the user of me track _ treatment of the audio sound track 

may request a segment from the beginning to the selected m this fashion complements the invention's approach to the 

point, or from the selected point to the end. video. 

For these and other options, suitable user interfaces in S5 techniques for summarizing the video have been 

addition to the scroller 28 are provided in the user interface found t0 be highly intuitive for average user. That is, the 

of FIG. 3. users who have tried an implementation of the invention 

Optionally, as a safeguard, the user interface may include have not needed any training or explaining on how to 

a predetermined ceiling for the total amount of video to be interpret the images the system provides, 

requested, so that the user will not inadvertently request a 60 HOW THE STORYBOARD FILES ARE CREATED: 

large segment. For instance, if ten seconds is the threshold, it is preferred that the set of frames making up the 

the system optionally warns the user before processing a storyboard are automatically created at the time the video 

request for a segment exceeding ten seconds in duration. data is prepared for storage in the repo sitory jind are loaded 

Upon confirmation of the request, such as by clicking on a into the database along with the data. 

YES/NO button, the request is processed. 65 Story Board data files are prepared after the scene changes 

Still other techniques will suggest themselves to persons are identified, using known techniques. This can be done 

skilled in the GUI field, and such other techniques fall within either from compressed or from non -compressed video. 
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FILE SIZES AND SOME STATISTICS: 

Various videos have different structure depending on their 
content. Full featured movies have, for example, longer 
scenes than MTV music videos or commercials. 

The keyframes are displayed to the user by means of a 
standard HTML Web browser, or any other image viewer. 
Here, where only static images are displayed, the average 
reduction in size of the data in the story board, relative to 
that of a MPEG-1 file, is about a factor of 100, 

Here are statistics from clips of four feature movies and 
from four IBM commercials. 

Full feature movies, measured by analyzing one clip from 
each of the 4 movies, average clip being about 30 sec. 

Keyframe size for SB is 12 KBytes (color). 

Average reduction in size MPEG-1 size/MB size=19 
times. 

Average reduction in size MPEG-1 size/SB size-127 
times. 

Average number of keyframes per second 0.38. 
NOTE: This data heavily depends on the content. Scenes 
with more action will have more keyframes. 

IBM Commercials, 4 full commercials analyzed, average 

length 41 sec. 
Average reduction in size MPEG-1 size/MB size»14 
Average reduction in size MPEG-1 size/SB size=64 
Average number of keyframes per second 0.24. 
The numbers, obviously, vary with content, but these 

provide an order of magnitude, which is believed to have 

wide applicability in a variety of different types of video 

content. 

Assuming, for the sake of illustration, -that_a keyframe 
^nt&rval a verages in th e- range of 2-4 seconds, for one minute 
" oT ~vIdeo, the story board SB p r< 7Yir* pc ahmit iO*n Vpy- 
ffTmes. it' each oi the keyframes is color, and requi res a 
tffpicaTvalue otatort- l lMlfjfrffgg^bcf^ e total data Jot the 
entire stdTjThojirrL SB k i ah ou jnkfc gpjCbytes per minute 
of video. The amount of time ^tiS sfer these keyframes, 
and the number of HTML pages needed to do so, may 
readily be estimated from this data total, given the remote 
link data transfer rate. 

SUMMARY AND CONCLUSION 

Using the foregoing specification, the invention may be 
implemented using standard programming and/or engineer- 
ing techniques using computer programming software, 
firmware, hardware or any combination or subcombination 
thereof. Any such resulting program(s), having computer 
readable program code means, may be embodied or pro- 
vided within one or more computer readable or usable media 
such as fixed (hard) drives, disk, diskettes, optical disks, 
magnetic tape, semiconductor memories such as read-only 
memory (ROM), etc., or any transmitting/receiving medium 
such as the Internet or other communication network or link, 
thereby making a computer program product, i.e., an article 
of manufacture, according to the invention. The article of 
manufacture containing the computer programming code 
may be made and/or used by executing the code directly 
from one medium, by copying the code from one medium to 
another medium, or by transmitting the code over a network. 

An apparatus for making, using, or selling the invention 
may be one or more processing systems including, but not 
limited to, a central processing unit (CPU), memory, storage 
devices, communication links, communication devices, 
servers, I/O devices, or any subcomponents or individual 
parts of one or more processing systems, including software, 
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firmware, hardware or any combination or subcombination 
thereof, which embody the invention as set forth in the 
claims. 

User input may be received from the keyboard, mouse, 
pen, voice, touch screen, or any other means by which a 
human can input data to a computer, including through other 
programs such as application programs. 

One skilled in the art of computer s cience will ea sil y he 
abl e to combine t he software created as described wit h 
appropriate ge neral umpuse m special ptlrpostT computer 
ha rdball tu tuale a computer s ystem and/or computer 
sub components embodying tne invention and to create a 
computer system and/or computer subcomponen ts for c ar- 
rying ouTthe method o f the invention . WhiieinTpreferred 
embodimenl^fThe present inventiCn~hus-been illustrated in 
detail, it should be apparent that modifications and adapta- 
tions to that embodiment may occur to one skilled in the art 
without departing from the spirit or scope of the present 
invention as set forth in the following claims. 

While the preferred embodiments of the present invention 
have been illustrated in detail, it should be apparent that 
modifications and adaptations to those embodiments may 
occur to one skilled in the art without departing from the 
scope of the present invention as set forth in the following 
claims. 

What is claimed is: 

1. A system for facilitating user examination and brows- 
ing of video data object information pertaining to a stored 
video data object (herein denoted "Object"), the Object 
including a beginning point, an end point, and points 
therebetween, the system comprising: 

an information processing apparatus having a user inter- 
face unit and a coupling unit for coupling to a video 
repository having video data objects, including the 
Object, stored therein; 

a sending unit for sending, through the user interface unit 
and over the medium, a request to view the Object; 

a receiving unit, responsive to receipt of a response to the 
request to view, for receiving a data file of reduced-size, 
thumbnail images of video stills and related audio 
portions from the respective points of the Object, each 
thumbnail image corresponding with one of the points 
in the Object; 

a viewing unit for displaying the thumbnail images prior 
to a portion of said Object being downloaded; 

an audio unit for playing said related audio portions with 
said display of said thumbnail images prior to said 
portion of said Object being downloaded; 

a user-operable selecting unit for selecting an interval in 
terms of the points, the interval delimiting the begin- 
ning and end of said portion of the Object which the 
user desires to view; and 

a downloading unit, operable responsive to operation of 
said selecting unit for downloading and providing the 
portion of the Object within the selected interval; 

said viewing unit for facilitating user viewing of the 
downloaded portion. 

2. A system as recited in claim 1, wherein: 

the video repository is located locally to the system; and 
said coupling unit includes a local coupling connection. 

3. A system as recited in claim 1, wherein said coupling 
unit includes a communication interface for coupling with a 
network communication medium, the medium being 
coupled to the video repository, the video repository being 
located at a location remote from the information processing 
apparatus. 
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4. A system as recited in claim 3, wherein: 

the user interface unit includes a computer having a World 
Wide Web browser; and 

the communication interface includes an Internet inter- 
face. 5 

5. A method for facilitating user examination and brows- 
ing of video data object information pertaining to a stored 
video data object (herein denoted "Object"), the Object 
including a beginning point, an end point, and points 
therebetween, the method to be practiced by an information 10 
processing apparatus having a user interface unit and a 
coupling unit for coupling to a video repository, wherein 
said video repository has said video data objects, including 
the Object, stored therein, the method comprising: 

sending, through the user interface unit and over a 5 
medium, a request to view the Object; 

responsive to receipt of a response to the request to view, 
receiving a data file of reduced-size, thumbnail images 
of video stills and related audio portions from the 
respective points of the Object, each thumbnail image 20 
corresponding with one of the points in the Object; 

playing said related audio portions while displaying said 
thumbnail images on said user interface unit prior to a 
portion of said Object being downloaded; 

receiving, from a user, a selection of an interval in terms 25 
of the points, the interval delimiting the beginning and 
end of said portion of the Object which the user desires 
to view; 

responsive to the selection of an interval, downloading 
and providing the portion of the Object within the 30 
selected interval; and 

facilitating user viewing of the downloaded portion. 

6. A system as recited in claim 1, wherein said displaying 
unit is also for displaying the thumbnail images in an array, 
the thumbnail images being in positions in the array related 35 
to their respective positions within the Object. 

7. A system as recited in claim 6, wherein: 

the number of thumbnail images from the data file 
exceeds a display capacity of the user interface unit; 

the thumbnail images are scrollably displayed on the user 40 
interface unit; and 

said displaying unit is also for displaying a user- 
manipulable user interface tool for allowing the user to 
control scrolling of the displayed thumbnail images. 

8. A system as recited in claim 1, wherein: 

the sequence of representations includes representations 
of scene change points within the Object and represen- 
tations of points within scenes of the Object; and 

the system further comprises a setting unit for setting a 5Q 
parameter giving a number of representations within 
scenes to be displayed. 

9. A system as recited in claim 1, wherein said receiving 
unit is also for receiving a data file of images of video stills 
from the respective points of the Object; and ss 

said system further including a displaying unit for dis- 
playing the images on the user interface unit. 

10. A system as recited in claim 9, wherein said displaying 
unit is also for displaying the images successively in time, 
whereby the user is able to perceive a flow of action in the 60 
Object. 

11. A system as recited in claim 10, wherein said display- 
ing unit is also for displaying the images one of: 

at a fixed time interval; 

at a time interval related to a real time interval, within the 65 

Object, between successive images; and 
at a rate controlled by the user. 



12. A system as recited in claim 10, wherein said display- 
ing unit includes a user-manipulable user interface tool for 
allowing the user to control a time interval between succes- 
sively displayed images. 

13. A system as recited in claim 10, wherein said audio 
unit is also for playing an audio sound track along with 
successively displayed images. 

14. A system as recited in claim 1, wherein said selecting 
unit is also for selecting desired ones of the representations 
by mouse clicks. 

15. A system as recited in claim 14, wherein said selecting 
unit is also for selecting first and second ones of the 
representations, which represent the beginning and end, 
respectively, of the desired portion of the Object. 

16. A system as recited in claim 14, wherein said selecting 
unit is also for selecting one of the representations, the 
desired portion of the Object being specified in terms of one 
of: 

(i) from the beginning of the Object to a selected 
representation, 

(ii) from the selected representation to the end of the 
Object, 

(iii) from the selected representation to an immediately 
succeeding one of the representations, and 

(iv) from the selected representation forward for a speci- 
fied amount of time. 

17. A system as recited in claim 1, wherein said viewing 
unit includes user-operable controls for at least one of (i) 
playback, (ii) fast -forwarding, (iii) reverse playback, (iv) 
fast reverse playback, and (v) pause. 

18. A computer program product, for use with an infor- 
mation processing apparatus having a user interface unit and 
a coupling unit for coupling to a video repository having 
stored video data objects, including a stored video data 
object (herein denoted "Object"), stored therein, for facili- 
tating user examination and browsing of video data object 
information pertaining to the Object, the Object including a 
beginning point, an end point, and points therebetween, the 
computer program product comprising: 

a computer-usable medium; 

a set of directions, provided on the medium, for directing 
the information processing apparatus to send, through 
the user interface unit, a request to view the Object; 

said directions, operable responsive to receipt of a 
response to the request to view, are also for directing 
the information processing apparatus to receive a data 
file of reduced-size, thumbnail images of video stills 
and related audio portions from the respective points of 
the Object, each thumbnail image corresponding with 
one of the points in the Object, and to play said related 
audio portions while displaying said thumbnail images 
on said user interface unit prior to a portion of said 
Object being downloaded; 

said directions, operable responsive to user command, are 
also for directing the information processing apparatus 
to select an interval in terms of the points, the interval 
delimiting the beginning and end of said portion of the 
Object which the user desires to view; 

said directions, operable responsive to operation of the 
information processing apparatus selecting an interval, 
are also for downloading and providing the portion of 
the Object within the selected interval; and 

said directions are also for directing the information 
processing apparatus to facilitate user viewing of the 
downloaded portion. 
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19. A method as recited in claim 5, wherein: 33. A method as recited in claim 31, wherein the selecting 
the video repository is located locally to the system; and includes selecting one of the representations, the desired 
said coupling unit includes a local coupling connection. P ortion of ^ 0b i ect bein g specified in terms of one of: 

20. A method as recited in claim 5, wherein said coupling (i) from the beginning of the Object to a selected 
unit also includes a communication interface for coupling 5 representation, 

with said medium, the medium being coupled to the video (ii) from the selected representation to the end of the 

repository, the video repository being located at a location Object, 

remote from the information processing apparatus. from the elected representation to an immediately 

21. A method as recited in claim 20, wherein: succeeding one of the representations, and 

the user interface unit includes a computer having a World (iv) from the representation forward for a speci- 

Wide Web browser; and fied amount of time 

the communication interface includes an Internet inter- 34. A method as recited in claim 5, wherein the facilitating 

face. of user viewing includes providing user-operable controls 

22. A computer program product as recited in claim 18, J5 f or a t least one of (i) playback, (ii) fast-forwarding, (iii) 
wherein said directions are also for directing the information reverse playback, (iv) fast reverse playback, and (v) pause, 
processing apparatus to select desired ones of the represen- 35, a computer program product as recited in claim 22, 
tations by mouse clicks. wherein said directions are also for directing the information 

23. A method as recited in claim 5, wherein the displaying processing apparatus to select first and second ones of the 
includes displaying the thumbnail images in an array, the ^ representations, which represent the beginning and end, 
thumbnail images being in positions in the array related to respectively, of the desired portion of the Object. 

their respective positions within the Object. 35. A computer program product as recited in claim 35, 

24. A method as recited in claim 23, wherein: wherein: 

the number of thumbnail images from the data file me video repository is located locally to the system; and 

exceeds a display capacity of the user interface unit; ^ sM CQUpling ^ includes a ^ connection , 

the thumbnail images are scrollably displayed on the user 37 A computer program product as recited in claim 18, 

interface unit; and wherein said coupling unit includes a communication inter- 

the displaying further includes displaying a user- face for coupling with a network communication medium, 

manipulable user interface tool for allowing the user to the network communication medium being coupled to the 

control scrolling of the displayed thumbnail images. 30 video repository, the video repository being located at a 

25. A method as recited in claim 5, wherein: location remote from the information processing apparatus, 
the sequence of representations includes representations 38. A computer program product as recited in claim 37, 

of scene change points within the Object and represen- wherein: 

tations of points within scenes of the Object; and the user interface unit includes a computer having a World 

the method further comprises setting a parameter giving a 35 Wide Web browser; and 

number of representations within scenes to be dis- tne communication interface includes an Internet inter- 

Played. f ace . 

26. Amethod as recited in claim 5, wherein the receiving 39, a computer program product as recited in claim 22, 
and displaying of said sequence includes: ^ wherein said directions are also for directing the information 

receiving a data file of images of video stills from processing apparatus to select one of the representations, the 

respective points of the Object; and desired portion of the Object being specified in terms of one 

displaying the images on the user interface unit. of: 

27. Amethod as recited in claim 26, wherein the display- (i) from the beginning of the Object to a selected 
ing includes displaying the images successively in time, 45 representation, 

whereby the user is able to perceive a flow of action in the ^ from the representation to the end of the 

Object. 05 j ecl> 

28. Amethod as recited in claim 27, wherein the display- ..... . , t , . . 

' - r J (m) from the selected representation to an immediately 

ine includes displaying the images one 01: v/ **■ j 

& r 3 & & succeeding one of the representations, and 

at a fixed time interval; so /. w ^ 1 . * c j *■ 

, , , A . i . . , . . A , . iL (iv) from the selected representation forward for a speci- 

at a time interval related to a real time interval, within the v L A n t * t - A 

„ . , . . tied amount 01 time. 

Object, between successive images; and 4Q A computer program product as recited in claim 18 

at a rate controlled by the user. wherein said directions are also for directing the information 

29. Amethod as recited in claim 27, wherein the display- processing a pp ara tu S to display the thumbnail images in an 
ing farther includes displaying a user-manipulable user 55 the thum5nail images 5eing in positions in the array 
interface tool for allowing the user to control a time interval related [0 ^ respective positions within the Object, 
between successively displayed images. 41. A computer program product as recited in claim 40, 

30. A method as recited in claim 27, further comprising wherein* 

the playing an audio sound track along with the successively * , . c 

disDlaved imaees 60 mc naimDer of thumbnail images from the data file 

31. A method as recited in claim 18, wherein the selecting cxcccds a dis P la y ca P acit y of mc USer mtcrface unit i 
of said interval includes selecting desired ones of the rep- the thumbnail images are scrollably displayed on the user 
resentations by mouse clicks. interface unit; and 

32. A method as recited in claim 31, wherein the selecting said directions are also for directing the information 
includes selecting first and second ones of the 65 processing apparatus to display a user-manipulable user 
representations, which represent the beginning and end, interface tool for allowing the user to control scrolling 
respectively, of the desired portion of the Object. of the displayed thumbnail images. 
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42. A computer program product as recited in claim 18, 
wherein: 

the sequence of representations includes representations 
of scene change points within the Object and represen- 
tations of points within scenes of the Object; and 

said directions are also for directing the information 
processing apparatus to set a parameter giving a num- 
ber of representations within scenes to be displayed. 

43. A computer program product as recited in claim 18, 
wherein said directions are also for directing the information 
processing apparatus to receive a data file of images of video 
stills from the respective points of the Object; and 

for directing the information processing apparatus to 
display the images on the user interface unit. 

44. A computer program product as recited in claim 43, 
wherein said directions are also for directing the information 
processing apparatus to display the images successively in 
time, whereby the user is able to perceive a flow of action 
in the Object. 

45. A computer program product as recited in claim 44, 
wherein said directions are also for directing the information 
processing apparatus to display the images one of: 
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at a fixed time interval; 

at a time interval related to a real time interval, within the 

Object, between successive images; and 
at a rate controlled by the user. 

46. A computer program product as recited in claim 44, 
wherein said directions are also for directing the information 
processing apparatus to display a user-manipulable user 
interface tool for allowing the user to control a time interval 
between successively displayed images. 

47. A computer program product as recited in claim 44, 
wherein said directions are also for directing the information 
processing apparatus to play an audio sound track along with 
successively displayed images. 

48. A computer program product as recited in claim 18, 
wherein said directions are also for directing the information 
processing apparatus to provide, at the user interface unit, 
user-operable controls for at least one of (i) playback, (ii) 
fast-forwarding, (iii) reverse playback, (iv) fast reverse 
playback, and (v) pause. 

***** 
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